screencastService: Add rate control to VA-API H.264 pipelines

The vah264enc and vah264lpenc pipelines do not specify any rate control
parameters, causing the encoder to use its default bitrate. At 4K this
results in ~12 Mbit/s recordings — roughly 18x larger than the VP8
software fallback (~630 kbit/s) with no visible quality benefit for
typical screencast content.

All other pipelines in the PIPELINES array constrain quality explicitly:
openh264enc uses qp-max=26/qp-min=26, vp8enc uses max-quantizer=17 and
buffer-size=20000. Only the two VA-API pipelines are left unconstrained.

Add rate-control=vbr bitrate=2048 target-percentage=50 to both VA-API
pipelines. This produces ~1 Mbit/s average / 2 Mbit/s peak, which is
comparable to what the other encoders produce and works across both AMD
(which lacks CQP) and Intel VA-API implementations.

Before (4K, 5s recording): 11688 kb/s, 7.2 MB
After  (4K, 5s recording):  2472 kb/s, 1.6 MB

Closes: https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/9165
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/4171>

diff --git a/js/dbusServices/screencast/screencastService.js b/js/dbusServices/screencast/screencastService.js
index 117eb449407f191b0b755dc81a01b0488a7517d4..d70bbb15b221896316ceff53eaa8700941ebc6b3 100644 (file)
--- a/js/dbusServices/screencast/screencastService.js
+++ b/js/dbusServices/screencast/screencastService.js
@@ -36,7 +36,7 @@ const PIPELINES = [
         pipelineString:
             'capsfilter caps=video/x-raw(memory:DMABuf),format=DMA_DRM,max-framerate=%F/1 ! \
              vapostproc ! \
-             vah264lpenc ! \
+             vah264lpenc rate-control=vbr bitrate=2048 target-percentage=50 ! \
              queue ! \
              h264parse ! \
              mp4mux fragment-duration=500 fragment-mode=first-moov-then-finalise',
@@ -47,7 +47,7 @@ const PIPELINES = [
         pipelineString:
             'capsfilter caps=video/x-raw(memory:DMABuf),format=DMA_DRM,max-framerate=%F/1 ! \
              vapostproc ! \
-             vah264enc ! \
+             vah264enc rate-control=vbr bitrate=2048 target-percentage=50 ! \
              queue ! \
              h264parse ! \
              mp4mux fragment-duration=500 fragment-mode=first-moov-then-finalise',